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I. Real Party in Interest (37 C.F.R. §41.37(c)(l)(i)) 

The real party in interest in the present appeal is Microsoft Corporation, the 
assignee of the present application. 

II. Related Appeals and Interferences (37 C.F.R. §41. 37(c)(1)(H)) 

Appellants, appellants' legal representative, and/or the assignee of the present 
application are not aware of any appeals or interferences which may be related to, will 
directly affect, or be directly affected by or have a bearing on the Board's decision in the 
pending appeal. 

III. Status of Claims (37 C.F.R. §41.37(c)(l)(iii)) 

Claim 29 has been withdrawn. Claims 1-28, 30 and 31 stand rejected by the 
Examiner. The rejection of claims 1-28, 30 and 31 is being appealed. 

IV. Status of Amendments (37 C.F.R. §41.37(c)(l)(iv)) 

Proposed amendments to claims 1, 5, 9, 16 and 18 in Applicants' Reply to Final 
Office Action have not been entered. 

V. Summary of Claimed Subject Matter (37 C.F.R. §41.37(c)(l)(v)) 
A. Independent Claim 1 

Independent claim 1 relates to a computer system for classifying items. (See e.g., 
p. 4, line 24 - p. 14, line 9, and, Fig. 1). The system includes a computer system 
component that applies probabilistic dependency models, one for each of a plurality of 
categories, to an item to provide with respect to each of the plurality of categories an 
indication of whether the item belongs. (See e.g., p. 4, lines 25 - 29, p. 5, lines 7 - 1 1, p. 
19, lines 21 - 25, p. 20, line 24 - p. 21, line 5, and, Fig. 7). The probabilistic dependency 
models collectively employ outputs from a plurality of classifiers. (See e.g., p. 6, line 13 
- p. 6, line 28). The outputs employed by the probabilistic dependency models vary 
among the probabilistic dependency models. (See e.g., p. 6, line 29 - p. 7, line 7). 
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B. Independent Claim 5 

Independent claim 5 relates to a computer system for classifying items. (See e.g., 
p. 4, line 24 - p. 14, line 9, and, Fig. 1). The system includes a computer system 
component that applies a probabilistic dependency model to classify an item. (See e.g., 
Fig. 1, reference designator 108, p. 4, lines 25 - 29, p. 5, lines 7 - 1 1, p. 19, lines 21 - 25, 
p. 20, line 24 - p. 21, line 5, and, Fig. 7). The probabilistic dependency model contains 
dependencies on one or more classical outputs from a plurality of classifiers. (See e.g., p. 
6, line 13 - p. 6, line 28). The probabilistic dependence model further contains 
dependencies on one or more reliability indicators. (See e.g., p. 2, line 25 - p. 3, line 2, 
p. 7, line 8 -p. 8, line 10). 

C. Independent Claim 9 

Independent claim 9 relates to a computer system comprising a first computer 
system component that learns, from training examples, probabilistic dependency models 
for classifying items according to one or more reliability indicators together with classical 
outputs from a plurality of classifiers. (See e.g., p. 2, lines 22-24, Fig. 1, reference 
designator 106, p. 4, line 30 - p. 5, line 6, p. 6, lines 13 - 21, p. 6, line 29 - p. 8, line 10, 
Fig. 2, p. 14, line 9 - p. 15, line 27). 

D. Independent Claim 14 

Independent claim 14 is directed to a computer readable medium having computer 
executable instructions for performing steps comprising: implementing a plurality of 
classifiers adapted to receive and classify at least one item, the plurality of classifiers 
each generating a score related to classification of the at least one item; and, for each of 
one or more categories, facilitating classification, selection, and/or utilization of the at 
least one item with a probabilistic dependency model that employs one or more of the 
scores and, in addition, one or more reliability indicators. (See e.g., p. 5, lines 15-24, p. 
15, lines 16 - 18, p. 16, lines 8 - 9, p. 4, lines 25 - 29, p. 5, lines 7 - 11, p. 19, lines 21 - 
25, p. 20, line 24 - p. 21, line 5, and, Fig. 7, p. 6, line 13 - p. 6, line 28, p. 6, line 29 - p. 7, 
line 7, p. 2, line 25 - p. 3, line 2, and, p. 7, line 8 - p. 8, line 10). 
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E. Independent Claim 16 

Independent claim 16 relates to a system for classifying items that comprises 
means for determining a model that classifies the items based on a probabilistic approach 
that combines information about the items including one or more classical outputs of 
classifiers and one or more attributes of the items other than classical outputs of 
classifiers. (See e.g., Fig. 1, reference designator 106, p. 4, line 24 - p. 5, line 6, p. 6, line 
13 - p. 13, line 28, Fig. 2, p. 14, line 1 1 - p. 15, line 19). The system further comprises 
means for applying the model to classify the items. (See e.g., Fig. 1, reference designator 
108, p. 5, lines 7 - 14, p. 13, line 29 - p. 14, line 9, Fig. 3, p. 15, line 28 - p. 16, line 13, p. 
19, lines 21 -25). 

The means for limitations described above are identified as limitations subject to 
the provisions of 36 U.S.C. §1 12 16. The structures corresponding to these limitations 
are identified with reference to the specification and drawings in the above-noted 
parentheticals. 

F. Independent Claim 17 

Independent claim 17 is directed to a computer-readable medium having stored 
thereon a data structure useful in classifying items that comprises first data fields 
containing data representing an attribute to test, wherein the attributes represented include 
both classical classifier outputs and reliability indicators. (See e.g., Fig. 4, p. 16, lines 14 
- 26). The data structure further comprises second data fields corresponding to the first 
data fields and containing data representing values against which to compare the 
attributes. (See e.g., Fig. 4, p. 16, line 26 - p. 17, line 2). Additionally, the data structure 
comprises third data fields containing data representing classifier outcomes. (See e.g., 
Fig. 4, p. 17, lines 3-7). Finally, the data structure comprises fourth data fields 
facilitating determination of relationships among instances of the first, second, and third 
data fields, the relationships having a decision tree structure with the first and second data 
fields corresponding to decision nodes and the third data fields corresponding to leaf 
nodes. (See e.g., Fig. 4, p. 17, lines 8-14). 
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G. Independent Claim 19 

Independent claim 19 relates to a method of generating a classifier that comprises 
obtaining a set of training examples. (See e.g., Fig. 2, reference designator 202, p. 14, 
lines 9-14). The method further comprises applying a probabilistic approach that uses 
the training examples to develop a model that combines evidence to provide an output 
relating to whether an item belongs in a category. (See e.g.,, Fig. 2, reference designator 
204, p. 14, line 14 - p. 15, line 14). Next, the method comprises storing the model in a 
computer-readable media for use as a classifier (See e.g., Fig. 2, reference designator 214, 
p. 15, lines 14-18). The method further provides the evidence comprises one or more 
classical outputs of other classifiers and one or more attributes of the item other than 
classical outputs of classifiers. (See e.g., p. 2, line 19 - p. 3 line 2, p. 4, line 30 - p. 5, line 

6, p. 13, line 29 - p. 14, line 8, and, p. 19, lines 16 - 20). 

H. Independent Claim 24 

Independent claim 24 is directed to a method of classifying items that comprises 
applying probabilistic dependency models, one for each of a plurality of categories, to an 
item stored in computer readable format to provide an output relating to whether the item 
belongs in the category with respect to each of the plurality of categories. (See, e.g., Fig. 

7, p. 20, line 24 - p. 21, line 5). The probabilistic dependency models collectively contain 
dependencies on outputs from a plurality of classifiers. (See e.g., Fig. 1, p. 19, lines 16 - 
25). The outputs considered by the probabilistic dependency models vary among the 
probabilistic dependency models. (See e.g., p. 20, line 30 - p. 21, line 2). 

I. Independent Claim 27 

Independent claim 27 relates to a method of combining a plurality of classifiers to 
classify items that comprises sequentially applying tests to the items to obtain test results. 
(See e.g., Fig. 5, p. 20, lines 3 - 15). The method further comprises classifying the items 
based on the test results. (See e.g., Fig. 5, p. 20, lines 3-15). The sequence of tests 
applied varies among the items in that the outcome of one or more tests affects whether 
another test is applied, whereby the classifiers utilized vary depending on the items. (See 
e.g., Fig. 5, p. 3, lines 7 - 9, p. 5, lines 2 - 4, p. 7, lines 26 - 29 and, p. 20, lines 3 - 15). 
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VI. Grounds of Rejection to be Reviewed (37 C.F.R. §41.37(c)(l)(vi)) 

A. Claims 1-28, 30 and 3 1 stand rejected as being unpatentable under 35 
U.S.C. § 102(e) over Gjerdingen et al. (US 6,539,395). 

VII. Argument (37 C.F.R. §41.37(c)(l)(vii)) 

A. Rejection of Claims 1-28. 30 and 31 Under 35 U.S.C. S102(e) 

Claims 1-28, 30 and 31 stand rejected under 35 U.S.C. §102(e) as being 
anticipated by Gjerdingen et al. (US 6,539,395). Reversal of this rejection is respectfully 
requested for at least the following reasons. 

L Gjerdingen et al. fails to teach or suggest each and every 
limitation set forth in the subject claims. 

A single prior art reference anticipates a patent claim only if it 
expressly or inherently describes each and every limitation set 
forth in the patent claim. Trintec Industries, Inc. v. Top-U.S.A. 
Corp., 295 F.3d 1292, 63 USPQ2d 1597 (Fed. Cir. 2002); See 
Verdegaal Bros. v. Union Oil Co. of California, 814 F.2d 628, 631, 
2 USPQ2d 1051, 1053 (Fed. Cir. 1987). The identical invention 
must be shown in as complete detail as is contained in the ... 
claim. Richardson v. Suzuki Motor Co., 868 F.2d 1226, 9 USPQ2d 
1913, 1920 (Fed. Cir. 1989) (emphasis added). 

The subject invention generally relates to information management and in particular 
to a system and method for automatically classifying items, (p. 1, lines 6, 7). The invention 
provides meta-classifiers and systems and methods for building meta-classifiers. (p. 2, lines 
17, 18). Meta-classifiers are combinations of multiple classifiers, (p. 1, lines 23, 24). A 
meta-classifier provides a determination or indication of whether an item belongs in a 
particular category, (p. 2, lines 18, 19). The meta-classifiers apply a probabilistic approach 
to combining evidence regarding correct classification of items, (p. 2, lines 19-21). Thus, 
meta-classifiers in accordance with applicants' invention take the form of probabilistic 
dependency models, (p. 2, lines 21, 22). Using a set of training data and machine learning 
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techniques, the probabilistic dependency models are constructed to effectively utilize 
evidence that can include outputs of multiple classifiers, (p. 2, lines 22-24). Additionally, 
the probabilistic dependency models of the invention can consider additional evidence, such 
as one or more reliability indicators, (p. 2, lines 25, 26). 

To the contrary, Gjerdingen et al. discloses "[a] method for creating a database that 
allows content based searching in the music domain." See Abstract. Gjerdingen et al. 
employs feature vectors which are employed to compare music samples. (Col. 3, 18-20). 
The feature vectors of Gjerdingen et al. can include a vocal quality vector, a sound quality 
vector, a situational quality vector, a genre vector, an ensemble vector and an instrument 
vector. (Col. 12, line 21 - col. 14, line 35). A modeling module analyzes acquired data and 
performs a similarity computation. (Col. 15, lines 6, 7). The similarity computation 
determines the optimum function that can represent similarity between different music 
samples, based upon defined music attributes (i.e. feature vector values). (Col. 15, lines 6- 
11). 

A function Fij represents the distances between music sample i and j and may be 
illustrated as: 

WgDg + WeDe + WvDv + WtDt + WiDi 

where Wg, We, Wv, Wt and Wi are individual weights allocated to individual music spaces. 
(Col. 16, lines 26 - 32). The plural weights Wg, We, Wt and Wi are calculated such that SI 
and Fij are at a minimum distance from each other. (Col. 16, lines 32- 34). 

Function Fij may be fit using linear regression or by nonlinear regression techniques. 
(Co. 16, lines 39, 40). Other tools may be used to compute the weights shown and fit 
function Fij: Bayesian estimation techniques, neural network techniques, classification trees 
and hierarchical clustering. (Col. 16, line 45 - Col. 17, line 21). 

With regard to classification trees, Gjerdingen et al. discloses "[classification trees 
define a hierarchical or recursive partition of a set based on the values of a set of variables." 
(Col. 17, lines 40 - 42). "In the present case, the variables are the elements of plural feature 
vectors." (Col. 17, lines 42, 43). "A decision tree is a procedure for classifying music into 
categories according to their feature vector values." (Col. 17, lines 43 - 45). "Expert 
pairwise data 403 A may be used to define a satisfactory decision tree and then the tree may 
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be applied to a larger set of music." (Col. 17, lines 45 - 48). "This method partitions music 
samples into mutually exclusive categories, wherein music samples within each category are 
considered similar." (Col. 17, lines 48 - 50). 

In the Advisory Action mail July 13, 2004, the Examiner asserts "[a] 
classification tree is a plurality of classifiers since the classification tree has a plurality of 
classifications." Advisory Action at p. 2. Applicants' representative respectfully submits 
that the Examiner's assertion is incorrect. 

As set forth in MPEP §21 1 1.01, words of the claims must be given their plain 
meaning unless applicant has provided a clear definition in the specification. In re Zletz, 
893 F.2d 310, 321, 13 USPQ 2d 1320, 1322 (Fed. Cir. 1989). Further, claim terms are 
presumed to have the ordinary and customary meanings attributed to them by those 
skilled in the art. Sunrace Roots Enter. Co. v. SRAM Corp., 336 F.3d 1298, 1302, 67 
USPQ2d 1438, 1441 (Fed. Cir. 2003); Brookhill-Wilk 1, LLC v Intuitive Surgical, Inc., 
334 F.3d 1294, 1298, 67 USPQ2d 1 132, 1 136 (Fed. Cir. 2003). It is the use of the words 
in the context of the written description and customarily by those skilled in the relevant 
art that accurately reflects both the "ordinary" and the "customary" meanings of the terms 
in the claims. Ferguson Beauregard/Logic Controls v. Mega Systems, 350 F.3d 1327, 
1338, 69 USPQ 2d 1001, 1009 (Fed. Cir. 2003). 

As set forth in the specification of the subject application, a decision tree model is 
one example of a classifier - it is not a plurality of classifiers: 

There are many applications for automatic 
classification of items such as documents, images, and 
records. To address this need, a plethora of classifiers have 
been developed. Examples include a priori rule-based 
classifiers, such as expert systems, and classifiers based on 
probabilistic dependency models learned from trained data. 
Classifiers based on probabilistic dependency models 
include classifiers based on decision tree models, support 
vector machines, Bayesian belief networks, and neural 
networks. 

p. 1, lines 10-16. 
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a. Independent claims 1 and 24 

Independent claim 1 recites limitations of "a computer system component that 
applies probabilistic dependency models ... wherein the probabilistic dependency models 
collectively employs outputs from a plurality of classifiers". Similarly, independent claim 
24 recites limitations of" applying probabilistic dependency models ... wherein the 
probabilistic dependency models collectively contain dependencies on outputs from a 
plurality of classifiers" Gjerdingen et al does not disclose employing the outputs of a 
plurality of classifiers to form a probabilistic dependency model. 

Applicants' representative acknowledges that classifiers based on probabilistic 
dependency models include classifiers based on decision trees models, support vector 
machines, Bayesian belief networks, and neural networks (p. 1, lines 14-16). However, the 
disclosure of classifiers in Gjerdingen et al is limited to computation of weights and function 
fitting. Gjerdingen et al does not teach or suggest employing outputs from a plurality of 
classifiers to form the probabilistic classifier as set forth in independent claims 1 and 24. 

Moreover, in the Final Office Action mailed April 5, 2004, the Examiner's states that 
"applicant does not claim 'the combination of a plurality of classifiers to form the 
probabilistic classifier'". (Final Office Action at p. 5). Such contention is without merit as 
independent claim 1 clearly recites the limitation of "the probabilistic dependency models 
collectively employ outputs from a plurality of classifiers", and, independent claim 24 
recites "applying probabilistic dependency models ... wherein the probabilistic dependency 
models collectively contain dependencies on outputs from a plurality of classifiers" 

In view of at least the foregoing, it is readily apparent that Gjerdingen, et al 
neither anticipates nor suggests the subject invention as recited in independent claims 1 
and 24 (and claims 2, 3, 4, 25 and 26 which depend there from). Accordingly, this 
rejection should be reversed. 

b. Independent claims 5, 9 and 14 
Independent claim 5 is directed to a computer system for classifying items and recites 
a limitation of "a computer system component that applies a probabilistic dependency model 
to classify an item, wherein the probabilistic dependency model contains dependencies on 
one or more classical outputs from a plurality of classifiers and dependencies on one or 
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more reliability indicators", (emphasis added). 

Similarly, independent claim 9 is directed to computer system and recites a limitation 
of "a first computer system component that learns, from training examples, probabilistic 
dependency models for classifying items according to one or more reliability indicators 
together with classical outputs from a plurality of classifiers". Independent claim 14 is 
directed to a computer readable medium having computer executable instructions for 
performing steps comprising "implementing a plurality of classifiers adapted to receive and 
classify at least one item, the plurality of classifiers each generating a score related to 
classification of the at least one item; and for each of one or more categories, facilitating 
classification, selection, and/or utilization of the at least one item with a probabilistic 
dependency model that employs one or more of the scores and, in addition, one or more 
reliability indicators". (Emphasis added). 

"[Reliability indicators are, in a broad sense, attributes of the items being classified." 
(p. 2, line 27). "These attributes can include characteristics of an item, source of an item, and 
meta-level outputs of classifiers applied to the item." (p. 2, lines 28, 29). "In general, a 
reliability indicator provides an indication of a classifier's reliability in classifying certain 
groups of items." (p. 2, line 29 - p. 3, line 1). 

As discussed supra, Gjerdingen et al. does not disclose combining the outputs of a 
plurality of classifiers to form a probabilistic dependency model. Furthermore, Gjerdingen et 
al does not disclose employment of reliability indicators with regard to the combination of 
the plurality of classifiers. 

In the Final Office Action mailed April 5, 2004, the Examiner states that "applicant 
does not claim 'the combination of a plurality of classifiers to form the probabilistic 
classifier'". (Final Office Action at p. 6). The Examiner's position is incorrect. Independent 
claim 5 includes the limitation "the probabilistic dependency model contains dependencies 
on one or more classical outputs from a plurality of classifiers and dependencies on one or 
more reliability indicators". Similarly, independent claim 9 is directed to computer system 
and recites a limitation "a first computer system component that learns, from training 
examples, probabilistic dependency models for classifying items according to one or more 
reliability indicators together with classical outputs from a plurality of classifiers". 
(Emphasis added). Finally, independent claim 14 is directed to a computer readable medium 
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having computer executable instructions for performing steps comprising "implementing a 
plurality of classifiers adapted to receive and classify at least one item. . ." (Emphasis 
added). 

In view of at least the above, it is readily apparent that Gjerdingen, et al. neither 
anticipates nor suggests the subject invention as recited in independent claims 5, 9 and 14 
(and claims 6, 7, 8, 10, 1 1, 12, 13, 15 and 30 which depend there from). Accordingly, 
this rejection should be reversed. 

c. Independent claim 16 
Independent claim 16 is directed to a system for classifying items and recites "means 
for determining a model that classifies the items based on a probabilistic approach that 
combines information about the items including one or more classical outputs of classifiers 
and one or more attributes of the items other than classical outputs of classifiers" . 
(Emphasis added). 

In the Final Office Action mailed April 5, 2004, the Examiner asserts that: 

To one of ordinary skill in the art, a classification tree will 
combine information about the items including one or more 
classical outputs of classifiers and one or more attributes of 
the items other than classical outputs of classifiers. The 
classification tree has various classifications and various 
nodes that are combined. The tree is the model. 

Final Office Action at p. 7. 

As noted supra, a decision tree model is one example of a classifier - it is not a 
plurality of classifiers, (p. 1, lines 10-16). Accordingly, the Examiner's contention is not 
correct. Gjerdingen et al. does not disclose employing the outputs of a plurality of 
classifiers to form a model. Furthermore, Gjerdingen et al. does not disclose employment 
of attributes with regard to the combination of the plurality of classifiers. 

It is readily apparent that Gjerdingen, et al. neither anticipates nor suggests the 
subject invention as recited in independent claim 16. Accordingly, this rejection should 
be reversed. 
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d. Independent claim 1 7 

Independent claim 17 is directed to a computer-readable medium having stored 
thereon a data structure useful in classifying items and recites: 

first data fields containing data representing an 
attribute to test, wherein the attributes represented include 
both classical classifier outputs and reliability indicators ; 

second data fields corresponding to the first data 
fields and containing data representing values against 
which to compare the attributes; 

third data fields containing data representing 
classifier outcomes ; 

fourth data fields facilitating determination of 
relationships among instances of the firs t, second, and third 
data fields, the relationships having a decision tree structure 
with the first and second data fields corresponding to 
decision nodes and the third data fields corresponding to 
leaf nodes. 

As discussed previously, Gjerdingen et al. does not disclose employing classifier outputs and 
reliability indicators to classify items. 

In the Final Office Action mailed April 5, 2004, the Examiner states "application 
does not claim 'combination classifier outputs and reliability indicators to classify items'". 
Final Office Action at p. 8. The Examiner's position is without merit as independent claim 
17 includes the limitation of "wherein the attributes represented include both classical 
classifier outputs and reliability indicators". (Emphasis added). 

In view of at least the above, it is clear that Gjerdingen, et al. neither anticipates 
nor suggests the subject invention as recited in independent claims 17 (and claim 18 
which depends there from). Accordingly, this rejection should be reversed. 
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e. Independent claim 19 

Independent claim 19 is directed to a method of generating a classifier and recites a 
limitation of "applying a probabilistic approach that uses the training examples to develop a 
model that combines evidence to provide an output relating to whether an item belongs in a 
category . . . wherein the evidence comprises one or more classical outputs of other 
classifiers and one or more attributes of the item other than the classical outputs of 
classifiers". (Emphasis added). Gjerdingen et al. does not disclose the employment of 
classifier outputs and attributes to classify items as in applicants' invention as recited in the 
subject claim. 

In the Final Office Action mailed April 5, 2004, the Examiner asserts that: 

To one of ordinary skill in the art, a classification tree will 
combine information about the items including one or more 
classical outputs of classifiers and one or more attributes of 
the items other than classical outputs of classifiers. The 
classification tree has various classifications and various 
nodes that are combined. The tree is the model. 

Final Office Action at p. 8. 

As noted previously, as set forth in the specification of the subject application, a 
decision tree model is one example of a classifier - it is not a plurality of classifiers, (p. 
1, lines 10-16). Accordingly, the Examiner's position is without merit. 

In view of at least the above, it is readily apparent that Gjerdingen, et al neither 
anticipates nor suggests the subject invention as recited in independent claim 19 (and 
claims 20, 21, 22 and 23 which depend there from). Accordingly, this rejection should be 
reversed. 

/ Independent claim 27 
Independent claim 27 is directed to a method of combining a plurality of classifiers to 
classify items and recites a limitation of "sequentially applying tests to the items to obtain 
test results. . . wherein the sequence of tests applied varies among the items in that the 
outcome of one or more tests affects whether another test is applied, whereby the 

13 



classifiers utilized vary depending on the items." (Emphasis added). Gjerdingen et al. does 
not disclose or suggest one or more tests affecting whether another test is applied as in the 
subject claimed invention. 

In the Final Office Action mailed April 4, 2004, the Examiner states "applicant does 
not claim in the body of the claim 'combination classifier outputs and reliability indicators to 
classify items.'" Final Office Action at p. 9. This is clearly incorrect - independent claim 27 
recites the limitation of "wherein the sequence of tests applied varies among the items in that 
the outcome of one or more tests affects whether another test is applied, whereby the 
classifiers utilized vary depending on the items". (Emphasis added). 

Gjerdingen, et al. neither anticipates nor suggests the subject invention as recited 
in independent claims 27 (and claim 28 which depends there from). Accordingly, this 
rejection should be reversed. 
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B. Conclusion 

For at least the above reasons, the claims currently under consideration are 
believed to be patentable over the cited references. Accordingly, it is respectfully 
requested that the rejections of claims 1-28, 30 and 31 be reversed. 

If any additional fees are due in connection with this document, the 
Commissioner is authorized to charge those fees to Deposit Account No. 50-1063 
(Reference No. MSFTP217US). 



AMIN & TUROCY, LLP 

24 th Floor, National City Center 
1900 East 9 th Street 
Telephone: (216)696-8730 
Facsimile: (216)696-8731 



Respectfully submitted, 

AMIN & TUROCY, LLP 




Himanshu S. Amin 



Reg. No. 40,894 
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VIII. Claims Appendix (37 C.F.R. §41.37(c)(l)(viii)) 

1 . A computer system for classifying items, comprising: 

a computer system component that applies probabilistic dependency 
models, one for each of a plurality of categories, to an item to provide with respect to 
each of the plurality of categories an indication of whether the item belongs; 

wherein the probabilistic dependency models collectively employ outputs 
from a plurality of classifiers; and 

the outputs employed by the probabilistic dependency models vary among 
the probabilistic dependency models. 

2. The computer system of claim 1 , wherein the dependency models 
collectively employ one or more reliability indicators. 

3. The computer system of claim 1, wherein the probabilistic dependency 
models are decision trees. 

4. The computer system of claim 1, wherein the items are texts. 

5. A computer system for classifying items, comprising: 

a computer system component that applies a probabilistic dependency 
model to classify an item, wherein the probabilistic dependency model contains 
dependencies on one or more classical outputs from a plurality of classifiers and 
dependencies on one or more reliability indicators. 

6. The computer system of claim 5, wherein the computer system outputs a 
quantitative measure relating to confidence that the item belongs in a category. 

7. The computer system of claim 6, wherein the probabilistic dependency 
models are decision trees. 
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8. The computer system of claim 6, wherein the items are texts. 

9. A computer system, comprising: 

a first computer system component that learns, from training examples, 
probabilistic dependency models for classifying items according to one or more 
reliability indicators together with classical outputs from a plurality of classifiers. 

1 0. The computer system of claim 9, further comprising a second computer 
system component that repeatedly invokes the first component to learn probabilistic 
dependency models employing various potentially effective reliability indicators and 
compares the performances of the resulting probabilistic dependency models to identify 
reliability indicators that are effective. 

1 1 . The computer system of claim 9, wherein the first computer system 
component employs the classical outputs from classifiers and the reliability indicators in 
the same manner. 

1 2. The computer system of claim 9, wherein the probabilistic dependency 
models are decision trees. 

13. The computer system of claim 9, wherein the items are texts. 

14. A computer readable medium having computer executable instructions for 

performing steps comprising: 

implementing a plurality of classifiers adapted to receive and classify at 
least one item, the plurality of classifiers each generating a score related to classification 
of the at least one item; and 

for each of one or more categories, facilitating classification, selection, 
and/or utilization of the at least one item with a probabilistic dependency model that 
employs one or more of the scores and, in addition, one or more reliability indicators. 
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1 5 . The computer readable medium of claim 1 4, wherein: 

the instructions implement a different probabilistic dependency model for 

each of two or more categories; 

the probabilistic dependency models are based on subsets of parameters 
selected from the group consisting of the scores and the reliability indicators; and 

the subsets of parameters vary among the probabilistic dependency 

models. 

1 6. A system for classifying items, comprising: 

means for determining a model that classifies the items based on a 
probabilistic approach that combines information about the items including one or more 
classical outputs of classifiers and one or more attributes of the items other than classical 
outputs of classifiers; and 

means for applying the model to classify the items. 

1 7. A computer-readable medium having stored thereon a data structure useful 
in classifying items, comprising: 

first data fields containing data representing an attribute to test, wherein 
the attributes represented include both classical classifier outputs and reliability 
indicators; 

second data fields corresponding to the first data fields and containing data 
representing values against which to compare the attributes; 

third data fields containing data representing classifier outcomes; 

fourth data fields facilitating determination of relationships among 
instances of the first, second, and third data fields, the relationships having a decision tree 
structure with the first and second data fields corresponding to decision nodes and the 
third data fields corresponding to leaf nodes. 

1 8. The computer-readable medium of claim 1 8, wherein the data represented 
by the first data fields comprises classical classifier outputs from a plurality of classifiers. 
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19. A method of generating a classifier, comprising: 
obtaining a set of training examples; 

applying a probabilistic approach that uses the training examples to 
develop a model that combines evidence to provide an output relating to whether an item 
belongs in a category; and 

storing the model in a computer-readable media for use as a classifier; 

wherein the evidence comprises one or more classical outputs of other 
classifiers and one or more attributes of the item other than classical outputs of 
classifiers. 

20. A method of identifying useful reliability indicators, comprising 
obtaining potentially useful reliability indicators; 

applying the method of claim 19 using various of the potentially useful 
reliability indicators as evidence; and 

comparing the resulting classifiers to identify which of the potentially 
useful reliability indicators are, in fact, useful. 

21 . The method of claim 19, wherein the model is a decision tree. 

22. The method of claim 19, wherein the evidence comprises classical outputs 
from two or more classifiers. 

23 . A method of classifying items, comprising: 
obtaining the items in computer readable format, 

employing a computer to classify the item using a classifier generated 
according to the method of claim 19. 

24. A method of classifying items, comprising: 

applying probabilistic dependency models, one for each of a plurality of 
categories, to an item stored in computer readable format to provide an output relating to 
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whether the item belongs in the category with respect to each of the plurality of 
categories; 

wherein the probabilistic dependency models collectively contain 
dependencies on outputs from a plurality of classifiers; and 

the outputs considered by the probabilistic dependency models vary 
among the probabilistic dependency models. 

25. The method of claim 24, wherein the dependency models collectively 
contain dependencies based on one or more reliability indicators. 

26. The method of claim 24, wherein the probabilistic dependency models are 
decision trees. 

27. A method of combining a plurality of classifiers to classify items, 
comprising: 

sequentially applying tests to the items to obtain test results; and 
classifying the items based on the test results. 

wherein the sequence of tests applied varies among the items in that the 
outcome of one or more tests affects whether another test is applied, whereby the 
classifiers utilized vary depending on the items. 

28. The method of claim 27, wherein one or more of the tests involves a 
reliability indicator. 

29. Withdrawn 

30. The computer system of claim 10, wherein the second component 
automatically selects the potentially effective reliability indicators. 

3 1 . The method of claim 23, wherein the items are texts. 
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IX. Evidence Appendix (37 C.F.R. §41.37(c)(l)(ix)) 

None. 

X. Related Proceedings Appendix (37 C.F.R. §41.37(c)(l)(x)) 

None. 
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